AI Guardrails Index:

We broke AI guardrails down to six categories.

We curated datasets and models that demonstrate the state of AI safety using LLMs and other open source models.

Introduction

Restricted Topics guardrails ensure AI interactions remain focused and relevant, similar to human professional interactions. They prevent AI systems from straying into unintended domains, politely redirecting off-topic queries. Essential in various applications like customer service, financial advising, and medical information systems, these guardrails enhance user experience, maintain conversation integrity, and minimize risks associated with off-topic discussions across diverse industries.

Results

Leaderboard
Metric:
Task:
DeveloperModelLatencyOutcome
Guardrails AIRestrict to Topic (LLM)
0.80 ms
0.76
Guardrails AIRestrict to Topic (BART)
0.12 ms
0.91
Guardrails AIRestrict to Topic (Hybrid)
0.82 ms
0.93
GoogleNLP Topic Detection
0.19 ms
0.83

Dataset Breakdown

DeveloperSamples
Beauty & Fitness > Face & Body Care > Clean Beauty
27
Computers & Electronics > Computer Hardware > Laptops & Notebooks
25
Finance > Accounting & Auditing > Billing & Invoicing
24
Hobbies & Leisure > Water Activities > Surf & Swim
24
Internet & Telecom > Mobile & Wireless > Mobile Phones
23
Autos & Vehicles > Classic Vehicles
21
Food & Drink > Beverages > Soft Drinks
21
Games > Card Games > Collectible Card Games
21
Health > Medical Facilities & Services > Doctors' Offices
21
Arts & Entertainment > Celebrities & Entertainment News
20
Business & Industrial > Construction & Maintenance > Civil Engineering
20
Jobs & Education > Education > Open Online Courses
17
See the full dataset here: Topic Restriction dataset

Conclusion

The Guardrails AI Ensemble model demonstrates superior performance for broad topic restriction, outperforming Google Natural Language in both general and specific classifications. While the ensemble model has higher latency, GPU acceleration significantly mitigates this drawback. The Guardrails AI Ensemble is particularly valuable for critical applications where performance is paramount, such as content moderation and compliance monitoring. For most real-world scenarios, its improved F1 score and comprehensive topic coverage outweigh the slight increase in processing time. When choosing between models, users should carefully balance F1 scores requirements against latency constraints, considering the specific needs of their application. Overall, the Guardrails AI Ensemble emerges as the preferred solution for robust and accurate topic restriction across a wide range of use cases.